A Conversational Avatar Framework for Digital Interactive StoryTelling

Digital Interactive Storytelling (DIS) is a relatively novel area of computer entertainment which aims at investigating interactive applications capable of generating consistent, emergent and rich stories. To provide new solutions for DIS, we have proposed a novel multi-agent DIS framework, DIEGESIS, which includes agents' coordination and new planning and replanning solutions for DIS. A natural development of DIS systems is to interface them with a 3D character animation framework, in which a 3D character would tell the story, or would be used for in-story character dialogues. In this paper, we propose to interface DIEGESIS with Charisma, our MPEG-4 based facial animation framework. We introduce DIEGESIS, explaining the planning algorithm for story generation, the story execution, and the agent coordination algorithms. We also introduce Charisma, a novel MPEG-4 facial animation framework for speech animation, which is also capable of recording and playback of MPEG-4 animation, alongside realistic dynamic lip sync animation based on Cohen-Massaro coarticulation model adapted to MPEG-4 facial animation (FA) specification. The preliminary experiments show that the coarticulation technique we have developed is suitable for storytelling and gives overall good and promising results when compared to related techniques. Keywords-component; Interactive Storytelling, Planning Algorithms, Facial Animation, Speech Animation, MPEG-4, Coarticulation